feat(tests): Add integration test framework, goldens for 4 APIs [gapic-generator-python] #905
Conversation
google-cla
bot
added
the
cla: yes
Codecov Report
@@ Coverage Diff @@
## master #905 +/- ##
=========================================
Coverage 100.00% 100.00%
=========================================
Files 26 27 +1
Lines 1608 1697 +89
Branches 328 347 +19
=========================================
+ Hits 1608 1697 +89
Continue to review full report at Codecov.
|
There was a problem hiding this comment.
Thanks @miraleung! My Bazel knowledge is lacking so I'll let other folks review.
There was a problem hiding this comment.
Overall looks good to me, but a few general questions/concerns:
-
It looks like a massive update, do we really need that many golden files? Can we pick like one API instead (with most feature coverage in it)? This is simply to reduce the number of goldens to maintain.
-
Most importantly this will change the development workflow - with every new feature or substantial modification, the golden tests most probably will have to be updated. Also, unfortunately, bazel is and alien technology for most of the Python developers (including gapic-generator-python developers), and making them to deal with unexpected nasty failures and necessity to run/update the tests in the bazel way can make them really unhappy. Basically I was under impression that if we do goldens for gapic-generator-python it must be done in the python-idiomatic way. Comparing to gapic-generato-java - the generator itself is build via bazel, so using bazel for goldens infra makes perfect sense. For python the things are a bit different. I'm afraid if we introduce the goldens in this current form, relatively soon the maintainers of the repo will remove them as very painful/non-idiomatic to maintain.
-
Even if we go with the goldens-via-bazel infra, is there some documentation speaking about how the development workflow is going to be changed by this, what developers need to run before posting the PR to make sure that the stuff passes and what to do if the goldens need to be updated (sorry if it is already there - i did try to all potential readme/.rst files in this PR but did not find documentation like that in those files).
|
Comments inline:
|
The relevant files to review are those ending in
.bzl. Any.pyfiles in this PR are generated goldens only.